Undergraduate Research Methods in Psychology
Department of Psychology
Don’t forget about the reading evidence - due only 1 week from today!
Continue work on the article critique and research proposal - we are now into the final stretch. Use the time in class to help cut down on the amount you have to do at home! (But still do work at home on these).
I will have office hours again at 3:00pm - 6:00pm EST in AuSable 1307 on Friday 11/22/2024. Please consider coming if you have recently been struggling or would like to discuss the projects. I am available via email as well.
We talked more about the many difficult pitfalls of internal validity in experimental designs, and how to reduce them!
We discussed how to cope with null results and possible causes for this result.
We talked about narratives and details in writing and critiquing research.
The Example: I want to explore the relationship between academic success and participation in a new support program, as I believe the program should result in a higher GPA. I recruit from the population of students on the honors track, who are all enrolled in the same general education classes for the next two semesters. I put up a poster with a QR code, asking for participants in the study, in order to recruit people. With my chosen participants, I randomly assign students into two separate groups, whether or not they participate in the program. Then, I record all students’ current GPA. I then start the new support program for half of the participants, and for those who are not in it, I do not do anything. After two semesters, I stop the program and re-measure all of the GPAs. I conclude that there is no difference in GPA between the students who do participate in the program, and those who do not. Throughout the study, I ensure that none of the students partake in other support programs, like tutoring or mentorship.
Putting up a poster asking for participants is effectively a form of convenience and self-sampling \(\rightarrow\) non-probabilistic sampling (as it is not based in randomness) \(\rightarrow\) low generalizability
(Everyone got points) Counter balancing of conditions is only necessary on within-subjects designs, as those are the type that are affects by order effects (i.e., exposure to one condition impacts the next one).
Explain why researchers combine independent variables in a factorial design.
Describe an interaction effect in both everyday terms and arithmetic terms.
Identify and interpret the main effects and interactions from a factorial design.
Understand and produce examples of when factorial design would be useful and/or appropriate
Discuss some basic statistics procedures that can be used with these designs
Up until now, we have only talked about experimental designs that deal with one manipulated/independent variable and one measured/dependent variable.
However, we have designs that can look at two (or more) IVs at once and see their individual and combined impact on the DV!
We refer to these as factorial designs.
We can add a second (and third) independent variable if we are curious about more than one.
In addition to the individual effects of both of the IVs, we also get an interaction effect that describes how they change each other’s relationship with the outcome.
Statistically, we might say this interaction is a “difference in differences”
When confronted with a causal relationship, sometimes we might say, “well it depends” - what it depends on is the second (or third) IV
We can see this even in our personal experiences, and many relationships do depend on other factors
Example: I am assessing how spicy I like my food (on a scale of 1 to 10; my outcome). First, is it cold or hot outside (IV 1)? Second, am I eating Thai or Italian (IV 2)? It is possible that my answer will be different based upon both of the IVs.
4 Possible Outcomes:
I like all of my food spicier when it is hot - Weather effect, but not food
Whether I like by food spicy or not depends on both the weather, and type of food - interaction effect
Specifically, we are looking to see whether we have a crossover interaction, like in the graph below:
When we work with more than one IV, we use a factorial design.
This creates more outcome unique conditions = # of Conditions in IV 1 x # of Conditions in IV 2 = total number of conditions
Both IVs do not have to be manipulated. Often, one will be some categorical, measured trait (e.g., gender, ethnicity, etc.)
In addition to our statistics, we should show these differences in plots! Interaction effects become especially clear with visual evidence.
Factorial designs can help us find whether outcomes are different for different types of people.
A strong intervention may not be as effective in a different group of people.
This can be a boon to our external validity, as we demonstrate findings in a more heterogeneous group.
We also can establish whether one variable appears to moderate another on the relationship with the outcome variable.
For some theoretical reasons, we may have good reason to believe that an effect differs based on some demographic variable.
Example: I have a new intervention meant to encourage flexibility in learning and taking in new content. However, I recognize that the neuroplasticity of older adults is just lesser in general. Therefore, I believe my intervention will likely be more effective for younger adults, than it will for older adults.
In essence, we may be able to add nuance and “it depends” to our hypotheses and investigate with factorial designs.
Just like with other experiments, we can lay out a factorial design as being between-groups or within-groups.
But, we can designate each variable as between or within, leading to a total of 3 possible designs:
This is when all IVs are between-groups (i.e., participants are arranged into entirely separate groups)
One nuance is that this will likely require the largest sample size, as each group will have about 1/4th the total number of participants
Much like with previous within-groups designs, this is when participants see every possible condition.
One thing to watch out for is the need for counterbalancing to prevent order effects
This is when one IV is between-groups, and the other is within-group.
This is fairly common if we have one demographic variable (between-groups) and one manipulated variable that both demographics are exposed to each level (within-groups).
Prof. Paul Moes: “God himself cannot interpret a 4-way interaction - neither can you”
We can do 3 IVs, but with each additional variable the interpretation becomes exponentially more difficult and complicated.
Remember to think carefully about what sorts of conclusions you can draw with a design before you use it, and whether an alternative provides a more parsimonious conclusion.
Week 12 Lecture - Complex Experiments || Undergraduate Research Methods in Psychology